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Abstract 


Decades  of  reform  have  been  largely  ineffective  at  improving  the  efficiency  of  the  DoD 
Acquisition  System,  due  in  part  to  the  complex  processes  and  stovepipe  activities  that  result  in 
duplication  of  effort,  lack  of  re-use  and  limited  collaboration  on  related  development  efforts. 

This  research  applies  Knowledge  Management  (KM)  concepts  and  methodologies  to  the  DoD 
acquisition  enterprise  to  increase  “Program  Self-awareness”  (Gallup  &  MacKinnon,  2008,  p.  2). 
This  research  supports  the  implementation  of  reform  initiatives  such  as  Capability  Portfolio 
Management  and  Open  Systems  Architecture,  which  share  the  common  objectives  of  reducing 
duplication  of  effort,  promoting  collaboration  and  re-use  of  components.  The  DoD  Maritime 
Domain  Awareness  (MDA)  Program  will  be  used  as  a  test  case  to  develop  prototype  data 
schemas  and  apply  text  and  data  mining  tools  to  identify  duplication  and/or  gaps  in  the  features 
of  select  MDA  technologies.  This  paper  will  also  provide  the  foundation  for  future  development 
of  the  Program  Self-awareness  concept  and  KM  tools  to  support  decision-making  and  improve 
the  effectiveness  of  the  DoD  Acquisition  System. 
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I 


Introduction 


A.  Background 

The  Department  of  Defense  (DoD)  fiscal  year  2009  budget  for  Research,  Development, 
Test  and  Evaluation  (RDT&E)  and  procurement  exceeds  $180  billion  (Gates,  2009,  p.  37). 

Given  such  huge  budget  outlays  and  the  increasing  pressures  of  shrinking  discretionary 
budgets  and  fragile  economy,  the  DoD  Acquisition  System  is  the  subject  of  intense  scrutiny  from 
government  oversight  activities,  industry,  and  the  general  public.  This  scrutiny  has  been 
amplified  by  highly  publicized  acquisition  program  failures,  continued  cost  and  schedule 
overruns  and  lengthy  development  cycles. 

DoD  acquisition  has  endured  an  environment  of  seemingly  perpetual  reform  to  arrest 
this  chronically  poor  performance,  resulting  in  complex  acquisition  process  models,  increased 
executive  oversight,  and  incremental  policy  changes.  The  effectiveness  of  acquisition  reforms 
has  yet  to  be  evidenced  in  the  overall  performance  of  the  DoD  Acquisition  System. 

Independent  and  government-chartered  studies  and  reports  have  repeatedly  highlighted  the 
need  for  improved  systems  engineering  and  business  processes  to  incorporate  best  practices 
from  the  commercial  sector. 

The  DoD  has  embraced  several  recommendations  from  these  critical  reports  and  moved 
to  adopt  several  commercial  best  practices  and  process  initiatives.  Two  such  policy  initiatives 
relevant  to  this  research  are  the  adoption  of  Capability  Portfolio  Management  (CPM)  and  Open 
Architecture  (OA)  approaches,  discussed  at  length  in  later  sections  of  this  paper.  CPM  and  OA 
are  relatively  early  in  their  implementation  and  address  different  levels  of  the  acquisition 
process,  but  reflect  the  overarching  DoD  goals  of  improving  decision-making  regarding 
systems-of-systems  (SoS)  acquisitions  to  avoid  duplication,  identify  gaps,  and  decrease  costs 
and  development  times. 

The  tools  and  processes  used  by  acquisition  decision-makers  to  support  implementation 
of  CPM  and  OA  are  not  well  defined.  A  fundamental  requirement  of  both  CPM  and  OA 
approaches  is  that  acquisition  managers  develop  an  awareness  of  related  efforts  and  activities 
across  an  enterprise  and/or  community  of  interest  (COI)  to  identify  duplication  of  effort, 
capability  gaps,  re-use  and  collaboration  opportunities.  It  is  the  premise  of  this  paper  that 
development  of  improved  “Program  Self-awareness”  is  fundamental  to  the  success  of  the  CPM 
and  OA  reform  initiatives.  This  paper  applies  commercial  and  government  best  practices  to 
develop  Program  Self-awareness  through  Knowledge  Management  (KM)  methods  and  tools. 

The  DoD  Maritime  Domain  Awareness  (MDA)  Program  will  be  used  as  a  test  case  for 
application  of  KM  decision  support  tools  to  provide  normalized  “views”  of  program  elements  and 
attributes,  termed  “features”,  to  support  informed  program  decision-making.  The  premise  of  this 
research  is  that  application  of  KM  tools  will  improve  Program  Self-awareness  and  support  the 
informed  decision-making  required  to  realize  the  full  potential  of  the  CPM  and  OA  initiatives. 

B.  Problem  Statement  and  Research  Question 

DoD  acquisition  is  an  extremely  complex  system  comprised  of  numerous  stakeholders 
and  organizations  that  navigate  an  array  of  procurement  processes  in  an  uncertain  environment 
to  deliver  useful  military  capability  to  the  warfighter  at  the  best  possible  value  to  the  government. 
Acquisition  reforms  have  been  largely  ineffective  at  improving  the  efficiency  of  the  system,  due 
in  part  to  stovepipe  activities  that  often  result  in  duplication  of  effort,  lack  of  re-use  and 
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collaboration  on  related  development  efforts.  This  research  applies  KM  concepts, 
methodologies,  and  tools  to  DoD  acquisition  programs  to  increase  its  self-awareness.  It  is  the 
goal  of  this  research  to  demonstrate  the  Program  Self-awareness  concept  through  application  of 
prototype  decisions  support  tools  to  the  DoD  MDA  Program  to  answer  the  following  research 
question. 


■  How  can  KM  methodologies  and  decision  support  tools  be  used  to  improve 
Program  Self-awareness  and  decision-making  to  reduce  duplication  and  enable 
collaboration  and  re-use  in  complex  DoD  acquisition  programs? 

C.  Methodology 

This  paper  provides  an  overview  of  ongoing  thesis  research  which  will  explore  the 
problem  of  duplication,  lack  of  re-use  and  collaboration  in  DoD  Acquisition  and  follow  the 
intuition  that  increased  “Program  Self-awareness,”  enabled  by  KM  decision  support  tools,  will 
improve  acquisition  process  efficiencies  in  these  areas.  The  research  will  be  grounded  in 
Systems  Theory  and  Congruence  Model  to  develop  an  understanding  of  the  DoD  Acquisition 
System  and  identify  root  causes  of  the  stated  problem.  This  research  will  apply  KM  tools  to  the 
DoD  MDA  Program  as  a  test  case  and  evaluate  the  potential  for  improved  Program  Self- 
awareness  based  on  feedback  from  the  office  of  the  DoD  Executive  Agent  (EA)  for  MDA.  This 
work  will  provide  the  foundation  for  future  research  on  the  Program  Self-awareness  concept  and 
development  of  KM  tools  with  the  goal  of  improving  decision-making  and  enabling  re-use  and 
collaboration  in  DoD  acquisition  programs 

D.  Scope 

The  impact  of  implementation  of  the  concepts  and  tools  suggested  in  this  research  on 
other  organizational  components  within  the  DoD  Acquisition  System  (structure,  processes, 
people)  are  not  addressed  in  depth  in  this  research.  It  is  recognized  that  further  research  will  be 
required  to  study  organizational  congruence  and  cultural  issues  to  realize  the  full  benefits  of  the 
Program  Self-awareness  concept. 

II.  Systems  Theory  and  Organizations 

This  research  explores  the  potential  for  change  in  the  DoD  Acquisition  System  through 
application  of  KM  tools  to  improve  Program  Self-awareness.  The  Congruence  Model,  depicted 
in  Figure  1,  is  grounded  in  Systems  Theory  and  provides  a  framework  to  understand  the 
complexity  of  the  DoD  Acquisition  System. 
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Formal 

Organization 


People 


Figure  1.  The  Congruence  Model 

(Mercer  Delta,  1998,  p.  14) 


This  research  focuses  on  the  potential  benefit  of  technology,  namely  KM  tools,  to 
improve  “fit”  among  acquisition  system  components  to  achieve  improved  output  efficiency  and 
facilitate  implementation  of  policy  objectives  such  as  CPM  and  OA.  The  Congruence  Model  is 
useful  in  this  context  as  it  highlights  the  interdependency  among  system  components,  which 
must  be  considered  when  introducing  such  tools  into  a  complex  system  (Mercer  Delta,  1998, 
pp.  1-15).  This  research  suggests  that  application  of  KM  tools  may  form  a  sort  of  “glue”  to 
improve  the  fit  among  components,  and  that  subsequent  change(s)  to  other  system 
components,  namely  organizations  and  processes  (work),  will  likely  be  necessary  due  to 
implementation  of  these  technologies 

This  research  seeks  to  demonstrate  the  potential  increase  in  MDA  Program  Self- 
awareness,  which  could  facilitate  improved  decision-making,  increased  collaboration,  object  re¬ 
use,  and  reduced  development  timelines.  Figure  2  applies  the  Congruence  Model  to  the  DoD 
Acquisition  System  and  highlights  the  opportunity  area  for  application  of  KM  tools  and 
collaboration  to  improve  fit  among  components  and  overall  efficiency  of  the  system. 
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Figure  2.  The  Congruence  Model  Applied  to  the  DoD  Acquisition  System 

(Mercer  Delta,  1998,  p.  14) 

III.  Program  Self-awareness 

This  research  defines  Program  Self-awareness  as  the  collective  and  integrated 
understanding  of  program  attributes  (system  technology  features,  R&D  activities,  etc.)  and 
surrounding  environment  by  program  decision-makers  (program  managers,  system  engineers, 
sponsors).  Program  Self-awareness  allows  decision-makers  to  recognize  relationships  among 
program  attributes  and  seize  collaboration  and  re-use  opportunities  to  support  cost  effective 
acquisitions. 

Achieving  Program  Self-awareness  in  complex  acquisition  programs  such  as  the  DoD 
MDA  program  is  a  lofty  goal  considering  the  myriad  of  stakeholders,  processes,  people, 
activities,  and  organizational  structures  involved.  This  research  will  highlight  the  potential  of  KM 
tools  to  provide  an  incremental  improvement  in  Program  Self-awareness.  The  figure  below 
represents  what  Program  Self-awareness  embodies  in  the  MDA  Community  of  Interest, 
supported  by  collaboration  and  use  of  KM  tools  to  enable  improved  decision-making  (Gallup  & 
MacKinnon,  2008,  p.  2). 
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Figure  3.  MDA  Program  Self-awareness 

(Gallup  &  MacKinnon,  2008) 

IV.  Knowledge  Management 

The  information  age  continues  to  shape  the  organizational  environment  and  produce 
varying  effects  on  all  system  components  of  the  Congruence  Model.  The  power  of  personal 
computing,  global  networking,  and  collaborative  technologies  are  now  fundamental  to  many 
organizational  processes — enabling  increased  speed,  availability,  and  volume  of  data  to  support 
decision-making.  These  technology  changes  have  challenged  organizational  norms  and  forced 
organizations  to  perform  varying  degrees  of  self-analysis  to  assess  the  impact  of  these  changes 
to  the  fit  among  organizational  components  (Mercer  Delta,  1998,  p.  15). 

The  challenges  posed  to  organizations  in  the  information  age  are  many,  to  include  the 
task  of  turning  massive  amounts  of  data  into  pertinent  knowledge  and  leveraging  the  potential  of 
the  network  enabled  “informal  organizations”  to  improve  decision-making.  The  study  of  the 
dynamics  and  potential  of  technology,  process,  and  structure  to  improve  organizational 
knowledge  and  decision-making  has  fueled  academic  study  and  technology  research  and 
development  under  the  umbrella  term  of  Knowledge  Management  (KM).  The  formal  definitions 
of  KM  vary  widely  among  theorists  and  practitioners  in  the  field,  but  generally  address  the 
common  goal  of  improving  the  ways  organizations  transform  data  into  knowledge  to  support 
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decision-making.  This  research  will  focus  on  how  KM  methodologies  and  tools  which  can  be 
applied  to  organizations  to  improve  process,  structure,  and  decision-making. 

The  application  of  KM  principles  to  DoD  acquisition  was  the  subject  of  research  by 
military  fellows  at  the  Defense  Systems  Management  College  (DSMC)  in  January  2000,  titled 
“Program  Management  2000:  Know  the  Way.  How  Knowledge  Management  Can  Improve  DoD 
Acquisition”  (Cho,  Hans  &  Landay,  2000).  The  DSMC  fellows  draw  the  following  conclusions 
relevant  to  this  research: 

■  the  commercial  sector  is  successfully  adopting  KM  strategies  to  achieve  competitive 
advantage; 

■  Implementation  of  KM  technologies  in  an  organization  must  consider  impacts  on  its 
people,  processes,  and  structure  to  be  successful; 

■  KM  initiatives  require  culture  change  and  must  have  the  full  support  of  the  leadership 
to  be  successful; 

■  Mangers  who  effectively  use  their  company’s  knowledge  were  able  to  overcome 
knowledge-based  barriers  and  institutional  stovepipes  to  improve  collaboration  and 
customer  relationships; 

■  KM  is  a  source  of  organizational  and  economic  value; 

■  Communities  of  Practice  or  Interest  (COP/COI)  are  forums  of  networked  people  with 
similar  interests  and  issues  which  come  together  to  address  problems,  provide 
solutions,  share  ideas,  and  build  communication  links.  COI  development  provides 
the  foundation  for  KM  implementation; 

■  KM  implementation  should  be  an  incremental  process  built  upon  small  successes. 
(Cho  et  al„  2000) 

Cho  et  al.  make  a  compelling  case  for  adoption  of  a  KM  concepts,  tools  and  strategy  in 
the  DoD  Acquisition  System.  This  research  will  apply  specific  KM  tools  to  a  specific  acquisition 
problem  in  hopes  it  will  lead  to  the  “small  success”  the  DSMC  researchers  suggest  is  vital  to 
foster  widespread  KM  adoption  in  DoD  acquisition. 

A.  KM  Tools 

KM  tools  and  methodologies  support  the  transformation  of  data  into  information  and 
knowledge.  The  KM  tools  relevant  to  this  research  include  data  and  text  mining,  data 
warehousing,  data  analysis  and  visualization. 

1.  Data  and  Text  Mining 

DoD  acquisition  programs  generate  massive  amounts  of  documentation  during  all 
phases  of  development  process,  to  include  text  documents,  spreadsheets,  and  structured 
relational  databases,  etc.  The  amount  of  data  and  text  contained  in  these  documents  is 
staggering  and  holds  great  potential  for  application  of  data  and  text  mining  techniques  to  derive 
and  discover  useful  information  that  can  be  used  to  generate  knowledge  and  improve  decision¬ 
making  from  a  sea  of  seemingly  unrelated  data. 

Data  mining  is  a  “class  of  information  analysis  based  on  databases  that  looks  for  hidden 
patterns  in  a  collection  of  data  which  can  be  used  to  predict  future  behavior.  True  data  mining 
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software  does  not  just  change  the  presentation,  but  actually  discovers  previously  unknown 
relationships  among  the  data”  (Turban,  Shardra,  Aronson  &  King,  2008,  p.  13). 

Text  mining  is  “the  application  of  data  mining  to  non-structured  or  less  structured  text 
files,  which  entails  the  generation  of  meaningful  numeric  indices  from  the  unstructured  text  and 
then  processing  those  indices  using  various  data  mining  algorithms”  (Turban  et  al. ,  2008,  p. 
224). 


This  research  will  apply  certain  data  and  text  mining  techniques  to  the  DoD  MDA 
Program  to  demonstrate  the  potential  for  increased  Program  Self-awareness  of  the  portfolio  of 
MDA  system  features  to  support  improved  programmatic  decision-making. 

2.  Data  Warehouses  and  Data  Marts 

Data  mining  techniques  require  a  set  of  data  be  defined  from  which  the  various  data 
mining  algorithms  can  be  applied  and  subsequent  analysis  be  performed.  This  set  of  data  is 
termed  a  data  warehouse  or  data  mart.  A  data  warehouse  is  a  “physical  repository  where 
relational  data  are  specifically  organized  to  provide  enterprise-wide,  cleansed  data  in  a 
standardized  format.”  (Turban  et  al.,  2008,  p.  223).  A  data  mart  can  be  considered  a  subset  of 
a  data  warehouse  which  can  be  used  to  support  a  functional  area,  department,  or  community  of 
interest.  These  terms  will  be  used  interchangeably  for  the  purposes  of  this  research  (Turban  et 
al.,  2008,  p.  222). 

The  development  of  data  warehouses  into  the  structured  form  required  to  support  data 
mining  is  not  a  trivial  process.  The  data  warehouse  will  need  to  be  developed  to  support  the 
functional  area  being  supported  and  have  the  following  fundamental  characteristics:  subject- 
oriented,  integrated,  time-variant,  and  nonvolatile.  The  data  warehouse  may  also  be  developed 
to  include  the  following  capabilities:  web-based,  relational/multi-dimensional,  client/server,  and 
include  metadata  (data  about  data.  (Turban  et  al.,  2008,  pp.  39-40). 

Text  mining,  on  the  other  hand,  is  focused  on  developing  new  meanings  and 
relationships  from  unstructured  data  in  the  form  of  documents  (memos,  e-mails,  instructions, 
policies,  etc.)  to  support  decision-making.  The  set  of  documents  required  to  support  text  mining 
can  vary  in  type  and  structure,  providing  much  more  flexibility  in  formulation  compared  to  data 
warehouse  development.  The  additional  benefit  of  text  mining  is  the  amount  of  information 
available  in  a  form  ready  for  processing,  which  includes  upwards  of  80%  of  the  data  a  typical 
organization  collects.  Text  mining  algorithms  are  also  complex  and  typically  involve  the 
following  steps. 

1.  Eliminate  commonly  used  words  (the,  and,  other); 

2.  Replace  words  with  their  stems  or  roots  (e.g.,  eliminate  plurals,  and  various 
conjugations  and  declarations); 

3.  Consider  synonyms  or  phrases  (e.g.,  student  and  pupil  may  be  grouped  together); 

4.  Calculate  the  weight  of  the  remaining  terms  (e.g.,  based  on  frequency  of  occurrence 
in  a  document  or  set  of  documents).  (Turban  et  al.,  2008,  pp.  159-160) 
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3.  Analytics  and  Visualization 

The  development  of  data  described  above  supports  its  transformation  to  information  and 
knowledge  through  the  process  of  analytics  and  visualization.  Analytics  can  be  defined  as  a 
“category  of  applications  and  techniques  for  gathering,  storing,  analyzing,  and  providing  access 
to  data  to  help  enterprise  users  make  better  business  and  strategic  decisions”  (Turban  et  al., 
2008,  p.  86).  This  research  will  apply  several  analytical  applications,  to  include  data  mining,  text 
mining  and  visualization  techniques  to  discover  new  information  and  knowledge.  These  KM 
tools  have  the  potential  to  highlight  relationships  among  program  “features”  to  support  decision¬ 
making  regarding  duplication  of  effort,  gaps,  re-use  and  collaboration  opportunities  in  the  DoD 
MDA  program.  For  the  purposes  of  this  research,  a  "feature"  is  a  marketable  behavior  or 
property  of  a  system,  ideally  documented  in  a  design,  such  as  the  “power  window”  feature  on 
modern  automobiles. 

B.  Collaboration 

This  research  has  repeatedly  identified  the  importance  of  collaboration  to  support  KM 
implementation.  The  DSMC  study  heavily  emphasized  the  linkage  between  KM  success  and  the 
organization’s  culture  of  information  sharing  and  collaboration.  The  DSMC  researchers  also 
concluded  that  a  typical  DoD  acquisition  program  performs  very  little  collaboration  across 
different  programs  other  than  informal  networks  of  functional  area  experts  formed  at  the  same 
physical  location.  When  development  teams  were  asked  how  often  they  go  outside  their 
program  organization  to  seek  knowledge  to  problems  they  faced,  the  most  frequent  response 
was  “rarely  if  ever.”  The  researches  found  it  wasn’t  that  the  teams  didn’t  recognize  the  potential 
power  of  collaboration,  they  just  “don’t  know  who  else  is  working  on  similar  issues  or  don’t  see 
any  connection  between  their  project  and  another  one  in  a  different  area”  (Cho  et  al.,  2000,  pp. 
1-4). 


The  size  of  the  DoD  Acquisition  enterprise,  lack  of  enterprise  collaboration  and  KM  tools 
and  stovepipe  organizational  structures  do  not  support  a  culture  of  information  sharing.  The 
continued  explosion  and  proliferation  of  networking  technologies  has  penetrated  the  DoD 
acquisition  environment  and  spawned  several  collaboration  and  knowledge-sharing  initiatives 
germane  to  this  research,  which  may  represent  the  early  stages  of  a  move  towards  greater 
collaboration  in  DoD  acquisition: 

In  recognition  of  the  imperative  and  potential  power  of  collaboration  to  support  the 
complex  DoD  Acquisition  System,  KM  and  acquisition  experts  at  NPS  (Thomas,  Hocevar  & 
Jansen,  2006)  studied  collaboration  in  the  most  complex  DoD  and  Interagency  acquisitions  to 
develop  a  “collaborative  capacity”  assessment  tool.  Figure  3  depicts  the  “Collaborative 
Capacity”  model  developed  by  Thomas  et  al  (2006)  to  guide  their  research.  The  notion  that 
collective  self-awareness  is  integral  to  the  success  of  solving  a  common  problem  can  be  derived 
from  this  model.  It  can  also  be  inferred  from  the  model  that  collaboration  is  the  “glue”  used  to 
bond  “stovepipe”  organizations  together  to  solve  a  common  problem  such  as  an  inter-agency 
acquisition. 
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Group  Organization 


Figure  4.  Collaborative  Capacity  Model 

(Thomas  et  al. ,  2006,  p.  7) 

V.  DoD  Acquisition  Initiatives 

Two  DoD  acquisition  policy  initiatives  relevant  to  this  research  are  the  adoption  of 
Capability  Portfolio  Management  (CPM)  and  Open  Architecture  (OA)  approaches.  Both  CPM 
and  OA  are  relatively  early  in  their  implementation  and  address  different  levels  of  the  acquisition 
process,  but  share  the  common  goal  of  improving  DoD  decision-making  regarding  systems-of- 
systems  (SoS)  acquisitions  to  avoid  duplication,  reduce  costs,  and  decrease  development 
times. 

A.  Open  Architecture  (OA) 

The  emphasis  on  open  systems  architecture  (OA)  has  increased  over  the  past  decade 
with  OA  now  recognized  as  an  integral  part  of  DoD  systems  engineering  and  acquisition 
processes.  OA  is  not  a  new  concept,  however,  and  draws  from  engineering  design  principles 
that  have  shaped  mature  industries  for  many  decades.  The  modern  automobile  is  one  such 
example  of  OA  design  principles,  as  it  supports  integration  of  thousands  of  its  components 
through  what  can  be  viewed  as  a  system-of-systems  design.  This  OA  design  allows  most 
components  to  be  built  by  numerous  manufactures  to  a  standard  interface  specification,  which 
allows  tires  built  by  numerous  manufactures  to  fit  onto  the  wheels  of  a  wide  range  of  vehicles. 
The  OA  approach  is  very  attractive  in  the  context  of  DoD  acquisition  as  it  offers  potential  for 
decreased  development  timelines  and  reduced  costs  largely  through  re-use  of  components  in 
system-of-systems  acquisitions.  OA  designs  also  support  quick  upgrades  and  modifications, 
removing  the  requirement  to  redesign  other  components  or  entire  system  as  would  be 
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necessary  due  to  change  propagation  in  closed  or  non-modular  system  designs.  The 
application  of  OA  to  the  design  of  software-intensive  systems  has  been  the  focus  of  early  OA 
initiatives,  to  include  the  Navy  PEO-IWS  Software  Hardware  Asset  Reuse  Enterprise  (SHARE) 
Repository,  which  serves  as  a  searchable  library  of  ship  combat  systems  software  and  related 
assets  available  for  re-use  by  eligible  contractors. (Johnson  &  Blais,  2008,  p.  1). 

The  increased  emphasis  on  OA  has  resulted  in  several  initiatives  to  establish  common 
technical  and  architectural  standards  to  promote  increased  re-use  and  interoperability  for  OA 
systems,  to  include  the  SHARE  repository  described  above.  These  efforts  are  critical  to  the 
success  of  DoD  OA  implementation  and  require  continued  development  of  common 
vocabularies  and  collaboration  tools  to  facilitate  discovery  of  related  efforts  and  potential  re-use 
opportunities. 

A  fundamental  requirement  of  OA  is  that  acquisition  managers  develop  an  awareness  of 
related  efforts  and  activities  across  an  enterprise  and/or  COI  to  support  decision-making 
regarding  re-use  and  collaboration  opportunities.  It  is  the  premise  of  this  paper  that 
development  of  Program  Self-awareness  is  fundamental  to  the  success  of  OA  policy  initiatives. 

B.  Capability  Portfolio  Management  (CPM) 

In  2006,  the  Deputy  Secretary  of  Defense  released  a  memorandum  to  introduce  the 
Capability  Portfolio  Management  (CPM)  approach  to  DoD  Acquisition.  The  intent  of  exploring 
the  CPM  approach  was: 

to  manage  groups  of  like  capabilities  across  the  (DoD)  enterprise  to  improve 
interoperability,  minimize  capability  redundancies  and  gaps,  and  maximize  capabilities 
effectiveness.  Joint  capability  portfolios  will  allow  the  Department  to  shift  to  an  output- 
focused  model  that  enables  progress  to  be  measured  from  strategy  to  outcomes. 
Delivering  needed  capabilities  to  the  joint  warfighter  more  rapidly  and  efficiently  is  the 
ultimate  criterion  for  the  success  of  this  effort.  (Deputy  Secretary  of  Defense,  2006,  p.  1) 

The  initial  implementation  of  CPM  included  establishment  of  four  capability  area  test 
cases  (Joint  Command  and  Control,  Joint  Net  Centric  Operations,  Battlespace  Awareness,  Joint 
Logistics)  to  evaluate  the  CPM  approach  with  the  long-term  goal  of  achieving  broader 
implementation  in  the  2009-2013  timeframe.  CPM  goals,  objectives,  and  guidance  emphasized 
the  importance  of  system-of-systems  engineering  approaches  and  “data  transparency”: 

test  case  managers-in  conjunction  with  existing  data  management  stewards  and  the 
Institutional  Reform  and  Governance  effort-should  work  together  to  establish  an 
approach  (business  rules,  data  structure  changes,  knowledge  management  tools)  that 
will  strengthen  the  linkage  of  authoritative  information  to  capabilities  without 
compromising  information  flexibility.  (Deputy  Secretary  of  Defense,  2006,  Attachment  A, 
P-4) 

CPM  implementation  was  further  directed  across  the  DoD  acquisition  enterprise  in  2008 
and  linked  to  all  nine  Tier  1  Joint  Capability  Areas  (JCA).  The  new  policy  detailed  CPM 
integration  and  alignment  with  existing  DoD  acquisition  structures  and  processes  to  achieve 
widespread  implementation.  (Deputy  Secretary  of  Defense,  2008,  p.  1 )  The  definition  of  CPM 
was  also  refined  to  “the  process  of  integrating,  synchronizing,  and  coordinating  Department  of 
Defense  capabilities  needs  with  current  and  planned  DOTMLPF  investments  within  a  capability 
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portfolio  to  better  inform  decision-making  and  optimize  defense  resources”  (Deputy  Secretary  of 
Defense,  2008,  Glossary,  p.  8). 

The  CPM  approach  is  relevant  to  this  research  in  that  it  is  grounded  in  improved 
acquisition  decision-making  to  reduce  duplication  of  effort  and  identify  capability  gaps  in  the 
DoD  portfolio  of  systems.  The  emphasis  on  development  supporting  data  structures,  KM  tools, 
and  implied  expectation  of  expanded  collaboration  provide  a  clear  linkage  between  DoD  policy 
and  this  area  of  research.  KM  tools  directly  support  CPM  decision-making  at  multiple  levels  of 
acquisition  as  will  be  demonstrated  with  the  DoD  MDA  Program  to  identify  relationships  among 
a  portfolio  of  system  features. 

VI.  MDA  Program 

The  National  Plan  to  Achieve  Maritime  Domain  Awareness  (MDA)  from  October  2005 
defines  the  Maritime  Domain  as  “all  areas  and  things  of,  on,  under,  relating  to,  adjacent  to,  or 
bordering  on  a  sea,  ocean,  or  other  navigable  waterway,  including  all  maritime-related  activities, 
infrastructure,  people,  cargo,  and  vessels  and  other  conveyances.”  Additionally,  it  defines  MDA 
as  “the  effective  understanding  of  anything  associated  with  the  maritime  domain  that  could 
impact  the  security,  safety,  economy,  or  environment  of  the  United  States.”  The  stakeholders  in 
this  enterprise  make  up  the  Global  Maritime  Community  of  Interest  (GMCOI),  which  includes 
“federal,  state,  and  local  departments  and  agencies  with  responsibilities  in  the  maritime  domain. 
Because  certain  risks  and  interests  are  common  to  government,  business,  and  citizen  alike, 
community  membership  also  includes  public,  private  and  commercial  stakeholders,  as  well  as 
foreign  governments  and  international  stakeholders”  (DHS,  2005,  p.  1). 

The  problem  set  that  faces  the  Navy,  a  key  member  of  the  GMCOI,  is  that: 

commanders  lack  access  to,  and  the  ability  to  process  and  disseminate,  the  broad 
spectrum  of  information  and  intelligence  that  enables  cooperative  analysis  necessary  to 
understand  maritime  activity  in  their  area  of  responsibility,  and  requisite  to  early  threat 
identification  and  effective  response  against  these  threats;  and  when  appropriate,  to 
enable  partners  to  respond  (Chief  of  Naval  Operations,  2009). 

Navy  MDA  is  key  to  addressing  this  problem  set  because  it  will  “enable  the  warfighter  to 
sustain  decision  superiority  to  successfully  execute  its  missions.  MDA  is  fundamental  to 
decision  making  superiority  at  all  levels  of  command”  (Chief  of  Naval  Operations,  2009).  The 
Navy  plans  to  improve  the  following  capabilities  to  achieve  MDA;  “focused  data  collection; 
technological  enhancements;  greater  cooperative  information  sharing;  supporting  enduring  and 
emerging  maritime  security  partnerships;  and  the  professional  development  of  navy  personnel 
within  the  maritime  operations  centers  at  naval  components  and  numbered  fleets”  (Chief  of 
Naval  Operations,  2009). 

VII.  MDA  Program  Self-awareness  Test  Case 

The  MDA  Program  is  indicative  of  complex  system-of-systems  acquisition  efforts  being 
undertaken  by  the  DoD.  The  MDA  program  includes  additional  complexity  caused  by  the 
extensive  international  and  interagency  involvement,  which  exhibit  the  complexities  shown  in 
the  Collaborative  Capacity  Model  shown  in  Figure  3. 
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This  research  will  develop  and  examine  a  representative  data  mart  of  structured  and 
unstructured  program  and  policy  documents  from  members  of  the  GMCOI.  This  task  is 
especially  challenging  in  that  there  is  not  one  consolidated  repository  for  MDA-related 
programmatic  documentation.  This  data  will  be  collected  from  various  members  in  the  GMCOI 
closely  involved  in  MDA  systems  development  and  acquisition.  Data  and  text  mining  tools 
will  be  applied  to  the  MDA  Data  Mart  using  the  methodology  depicted  in  Figure  5  (Turban  et  al., 
2008,  p.  156). 


Figure  5.  Data-mining  Process  Recommended  by  CRISP-DM 

(Turban  etal.,  2008,  p.  156) 

To  date,  this  research  has  gathered  program  documentation  related  to  three  prototype 
MDA  systems,  to  include  Predictive  Analysis  for  Naval  Deployment  Activities  (PANDA),  a 
Defense  Advanced  Research  Projects  Agency  (DARPA)  project,  Track  Assessment  and 
ANomoly  Detection-Maritime  (TAANDEM)  software  subsystem,  and  Comprehensive  Maritime 
Awareness  System  being  developed  through  the  Navy  Research  Lab  (NRL).  These  documents 
have  been  placed  into  the  MDA  data  mart  for  use  in  our  modeling  and  analysis. 

The  next  step  in  our  research  will  be  to  further  our  data  understanding  and  prepare  the 
data  for  application  of  the  various  mining  algorithms.  This  phase  of  the  research  is  underway  as 
this  paper  is  being  prepared.  NPS  KM  research  expertise  and  cutting-edge  data  and  text 
mining  applications  will  be  leveraged  during  this  phase  of  the  research.  After  the  initial  data 
cleansing  and  preparation,  the  mining  tools  will  be  applied  to  the  data  mart  for  subsequent 
evaluation  and  analysis  using  visualization  products  to  identify  common  features,  capability 
gaps,  and  relationships  between  MDA  system  features.  We  expect  several  iterations  of  this 
process  to  extract  useful  data  from  the  models. 
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Using  preliminary  data  and  the  Quantum  Intelligence  (Ql)  data  and  text  tools  developed 
by  Dr.  Ying  Zhao,  the  visualization  products  depicted  in  Figures  6  and  7  were  developed  to 
demonstrate  representative  products  of  this  research  to  highlight  relationships  among  system 
feature  data. 
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Figure  6.  Sample  MDA  System  Cluster  Visualization 

(Zhao,  2009) 


DEFENSE  ACQUISITION  IN  TRANSITION 


-  156- 


“Fan  out”  with  “Fan  in” 
Illustration 

•  The  more  “arrows  out”  indicate 
more  features  shared  with 
another  technology 

•  Reveals  overlapping  or  missing 
common  features  (e.g.  EMIO) 

•  Works  like  a  “social  network” 


Figure  7.  Sample  MDA  System  Cluster  Visualization 

(Zhao,  2009) 

The  final  step  in  the  data  mining  process  is  deployment.  As  this  is  only  a  demonstration 
of  KM  tool  utility  for  Program  Self-awareness,  we  do  not  plan  to  deploy  the  algorithms 
developed  during  this  process.  This  work  will  for  the  foundation  for  a  larger  effort  by  the  DoD 
EA  for  MDA  that  will  hopefully  be  applied  to  a  much  larger  data  mart  developed  from  the  entire 
GMCOI. 


VIII.  Predicted  Findings 

The  MDA  Program  is  representative  of  complex  DoD  Acquisition  Programs.  KM 
concepts  and  tools  demonstrate  utility  for  improving  Program  Self-awareness  to  help  identify 
portfolio  gaps  and  duplication  which  can  lead  to  improved  resource  allocation  decision-making, 
collaboration  among  acquisition  activities,  and  re-use  of  SoS  components.  Figure  8  provides  an 
overview  of  the  research  methodology  described  above. 
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Collaboration  &  Reu 


Figure  8.  Program  Self-awareness  KM  Process 

As  mentioned  above,  a  central  repository  for  MDA  programmatic  documentation  does 
not  currently  exist.  This  research  will  recommend  development  of  a  GMCOI  MDA  web  portal  for 
use  as  a  data  warehouse  to  support  future  KM  implementation  and  to  promote  collaboration  and 
re-use.  We  hope  this  work  will  provide  foundation  for  future  work  to  refine  Program  Self- 
awareness  concept  and  KM  implementation  in  DoD  Acquisition. 
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Overview 


•  Knowledge  Management  (KM)  concepts  and  tools 
applied  to  improve  Program  Self  Awareness 

•  Benefits  Acquisition  and  RDT&E  program  managers 
and  supports  implementation  of  OA  and  CPM 


•  Sponsors 


-  NPS  Acquisition  Research  Program 

-  NPS  Distributed  Information  Systems  Experimentation 
(DISE)  Group 

-  Briefed  to  DoD  EA  for  Maritime  Domain  Awareness 
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Program  Self  Awareness 


•  The  collective  and  integrated  understanding 
of  program  attributes  (system  technology 
features,  R&D  activities,  etc)  and  surrounding 
environment  by  program  decision  makers 
(PMs,  system  engineers,  sponsors) 


Allows  decision  makers  to  recognize 
relationships  among  program  attributes  and 
seize  collaboration  and  re-use  opportunities 
to  support  cost  effective  acquisitions 


Self  Awareness? 
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Ref:  DAU,  2008,  DoD  5000  Brief 


Test  Case 


Maritime  Domain  Awareness  (MDA)  Program 

-  Environment  is  complex,  many  stakeholders, 
programs,  processes,  related  activities 

-  Self  Awareness  needed  to  coordinate  and  synchronize 
portfolio  of  activities  and  technologies 

-  Coherent  views  needed  to  develop  Self  Awareness 

-  Support  informed  decision  making  (resourcing  RDT&E 
efforts,  acquisition  strategies,  etc) 

-  Improve  developmental  efficiency  and  speed 
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MDA  Program  Self  Awareness 
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Methodology 


•  Apply  Congruence  Model  to  DoD  Acquisition 

-  Identify  potential  sources  of  poor  “fit”  which  lead  to 
duplication,  lack  of  re-use,  limited  collaboration 

•  Collect  structured  and  unstructured  program  data  of 
select  MDA  technologies 

•  Apply  KM  tools  (data  and  text  mining)  to  prototype  MDA 
Data  Mart  to  identify  “clusters”  of  MDA  system  features 


•  Analyze  data  and  visualizations  to  identify  system 
feature  clusters  which  enable  views  of  potential  gaps 
and  duplication  of  effort 
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The  Congruence  Model 


A  Model  To  Understand  Complex  Acquisition  Programs 
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Ref:  Mercer  Delta,  The  Congruence  Model,  1998 
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Data  Analysis 


•  Apply  KM  concepts  and  support  tools 

-  Data  repository 

-  Cluster  mining  techniques 

•  Group  MDA  technologies  based  on  their  features  derived 
from  structured  and  unstructured  data 


-  Visualization  and  analysis 


•  Leveraged  NPS  KM  expertise  to  evaluate  results 


-  Team  of  experts 

•  Program  and  KM  expertise 
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Structured  Data  Analysis 


•  Many  organizations  have  databases,  and  use  queries 
and  reports  to  extract  and  organize  data  for  decision 
making 

•  Text  mining  can  also  be  applied  to  structured  data  to 
highlight  previously  unknown  relationships 

•  MDA  Objective  data  from  FIRE  experimentation 
database 


Developed  visualization  of  feature  associations 
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Feature  cluster  relationships  to  MDA 

systems 
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Findings 


•  MDA  Program  is  representative  of  complex  DoD 
Acquisition  Programs 

-  Self  Awareness  needed  but  difficult  to  achieve 


-  Technology  not  being  leveraged  to  overcome 
information  stovepipes  and  cultural  barriers 


•  KM  concepts  and  tools 


-  Can  improve  Program  Self  Awareness  -  discovery  of 
program  relationships 


-  May  expose  portfolio  gaps  and  duplication  and 
promote  collaboration  and  component  re-use 

-  Team  of  KM  and  Program  experts  needed 
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Recommendations 


•  Acquisition  reform  efforts  must  invest  in  proven 
enterprise  technologies  (KM,  collaboration  tools) 

•  Develop  a  KM  implementation  strategy 

-  Incremental,  built  on  small  successes 

-  Implement  @  program  level  as  a  test  case 

-  Champion(s)  needed  to  drive  change  in  system 

•  Conduct  future  research 

-  Refine  KM  tools  -  drill  down,  visualization  products 

-  Applications  -  requirements  development,  traceability 


Questions 

and 

Discussion 
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BACK-UP 
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Methodology 


Observed  Problem  and 
Intuition 

Theoretical  POD  /  Gap 

Research  Question 

Data  Source 

Duplication  of  effort, 
limited  re-use  and 
collaboration  in  DoD 
Acquisition  Programs  due 
to  lack  of  Program  Self 
Awareness 

Systems  Theory  and 
Congruence  Model  -  a 
model  to  understand  the 
acquisition  environment 
and  Program  Self 
Awareness 

How  can  Knowledge 
Management  methods  and 
tools  be  used  to  improve 
Program  Self  Awareness, 
collaboration  and  re-use 
in  complex  acquisition 
programs? 

Case  study  of  Navy  MDA 
Program 

-Develop  MDA  Data  Mart 
from  structured  and 
unstructured  program 
data  sources 

Improved  DOD  Acquisition 
Program  efficiency  and 
effectiveness 

Better  understanding  of 

DoD  Acquisition  System 
knowledge  environment 
and  potential  for  improved 
Program  Self-awareness 
enabled  through  KM  tools 

-  Collaboration  complex 
and  not  efficient 

-  Lack  of  Program  Self 
Awareness  due  to 
complexity  and  culture 

-  KM  tools  can  be  applied 
to  improve  MDA  Program 
Self  Awareness  and 
decision  making 

-  Mine  Data  Mart  to  derive 
system  “feature”  data  and 
develop  visualization  tools 
to  show  relationships 
among  system  attributes 

-  Identify  duplication  and 
opportunities  for 
collaboration,  re-use  and 
efficiency 

Predicted  Impact 

Contribution 

Findings 

Data  Analysis 
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Congruence  Model 


A  Model  To  Understand  Complex  Acquisition  Programs 
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Collaborative  Capacity 


m  * 


Purpose  & 
Strategy 


Organization  A 


Structure 


Lateral 

Mechanisms 


wmm 


Interagency  Team 
Group  Organization 

Ref:  Figure  2  from  Building  Collaborative  Capacity  Paper  (NPS  ARP)  -  Lewin’s 
“force  field”  analysis  model  (McShane  &  Van  Glinow,  2005). 


The  Objective  -  Increased  Collaboration  to  solve  a  common  Problem 
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MDA  Program  Relations 
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Notes 
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Sponsor  ID 
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ContactID 


SystemID 

SystemName 

OtherSystemName 

SystemDescription 

MDATaskID 

JCASupported 

SystemRequirement 

DeveloperlD 

FundingSource 

FundingType 

TechReadinessLevellD 

SponsorlD 

ProjectBeginDate 

ProjectEndDate 

FeaturelD 

ContactID 


BriiFilPa 


Activity  ID 

ActivityName 

Activity  TypelD 

SponsorlD 

ActivityDescription 

Notes 

ContactID 


ContactID 

FirstName 

LastName 

Address 

City 

StateOrProvince 

PostalCode 

Country/Region 

CompanyName 
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MobilePhone 

Email 

ContactTypelD 


Thesis  Test 
Case 


A  Notional  MDA  Proqram  Data  Schema  to  support  Self  Awareness  and  Decision  Makinq 


Challenge  of  Collaboration 


DRIVING  FORCES 

RESTRAINING  FORCES 

DESIRED 

END 

RESULT 

1  > 

<  1 

— i 

Organization 

design 

component 

“Success"  factors 
that  contribute  to 
collaborative  capacity 

“Barriers"  that  inhibit 
collaborative  capacity 

Collaborative 

Purpose  & 
strategy 

-  “Felt  need”  to 
collaborate 

Common  goal  or 
recognized 
interdependence 
Adaptable  to 
interests  of  other 
organizations 

Divergent  goals 

Focus  on  local  organization 
over  cross-agency  (e.g., 
regional)  concerns 

Lack  of  goal  clarity 

Not  adaptable  to  interests  of 
other  organizations 

capacity  that  lead 
to  high 
performance 

Structure 

Formalized 
coordination 
committee  or  liaison 
roles 

Sufficient  authority 
of  participants 

Impeding  rules  or  policies 
Inadequate  authority  of 
participants 

Inadequate  resources 

Lack  of  accountability 

Lack  of  formal  roles  or 

Lateral 

mechanisms 


procedures  for  managing 
collaborate 


Social  capital  (i.e., 

interpersonal 

networks) 

Effective 

communication  and 

information 

exchange 

Technical 

interoperability 


Lack  of  familiarity  with  other 
organizations 

Inadequate  communication 
and  information  sharing 
(distrust) 


Ref:  Buliding  Collaborative  Capacity  Paper(N£S>^RP)  -  Lewin’s  “force 
field”  analysis  iiimiIi  I  il  ]cj  |i  11  1  in  Glinow,  2005). 


Barriers  to  Re-use 


Force  Field  Analysis  -  Services  Reuse 

Driving  Forces  [ 


Restraining  Forces 


- - -^-= 

Net-Centric  transformation  strategy^ 

unclear  business  incentives  &  ROI, 

Emerging  industry  standards 

Changing  standards  and  technologies 

Need  for  accelerated  capability  in  field 

No  easy  process  for  finding,  prioritizing  and 
testing  modifications  to  operational  systems 

DoD  Net-centric  Directives,  Instructions 
and  Strategies 

inadequate  trust  of  services  for  availability, 
reliability  and  supportability 

Certification  &  accreditation  reuse 

Security  concerns  and  lack  of  mechanisms  for 
certification  and  accreditation 

The  promise  of  service-oriented  and  net- 
centric  architectures 

Service  doesn't  completely  meet  users' 
needs,  different  semantics  across 
communities 

Reduced  development  time  and  cost 

Uncertain  load  balancing,  support  and 
governance 

Reduced  maintenance  cost 

Vendor  incompattiility 

Easier  exchange  of  data 

No  mechanism  to  discover  available  services 
and  data 

Ref:”  Dave  Chesebrough.The  Role  of  Architecture  in  Moving  DoD  to  a  Net-Centric  Environment”,  Software 

TechNews  Dec  06 
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Cluster  Visualization 


W  Graph 

Profiler  Predictor  Transition  Association  Search  Tree  Analysis 
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Notional  MDA 
Associations 
*  Illustrates  how 
many  features  two 
systems  share 
with  “strength”  of 
associations  (e.g. 
CMA  vs.  Global 
Trader) 


Ref:  Yina  Zhao.  Quantum  Intelliaence.  2  Mar  09 


